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DETAILED ACTION 

1. The amendment filed on 25 November 2005 have been received and entered. Claims 1- 
33 as amended are pending in the application. 

Claim Rejections - 35 USC §103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

2. Claims 1-4, 6-10, 12-15, 17-21, 23, 25-29 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Partanen et al. U.S. Publication 2003/0001899 (hereinafter "Partanen") and 
Skinner U.S. Patent 6,661,920. 

Referring to claims 1 and 18, Partanen teaches a method and system comprising user 
interface code evaluating a program field that has focus against information indicative of 
whether the field is configured to receive text input (the interface determines whether the user 
has initiated manuscript input to any point on a touch-activated screen of a display, thereby 
causing a semi-transparent window to be opened) (Partanen: page 1 , paragraph 00 1 6); and if the 
field is configured to receive text input (if a user activated manuscript input has been received) 
(Partanen: page 1, paragraph 0016), providing a visible user input interface at a displayed 
location relative to the field such that the user input interface is operable to receive handwritten 
data while the field is operable to receive input data (upon user initiation of manuscript input at 
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point 36 on the display screen of Figure 1, a semi-transparent window is displayed relative to the 
field, allowing users to input manuscript characters, for example, the letter "I", while the field is 
operable to receive input data such as user input of particular letters and/or numbers from a 
virtual keyboard) (Partanen: page 2, paragraph 0032 and page 3, paragraph 0038); receiving 
handwritten data at the input interface (Partanen: page 2, paragraph 0022 and Figure 1); 
providing the handwritten data to a recognition engine (relaying the pattern of the handwritten 
character to the handwriting recognition software) (Partanen: page 2, paragraph 0025); and 
returning a recognition result to the program (for example, recognizing the handwritten "I" and 
outputting it as the character "I" in Figure 1) (Partanen: page 2, paragraph 0022). However, 
although Partanen teaches a virtual keyboard, Partanen fails to explicitly teach simultaneously 
receiving typed and handwritten user input. Skinner teaches an interface that allows users to 
enter handwritten data similar to that of Partanen. In addition. Skinner further teaches receiving 
typed user input and the program field being operable to simultaneously receive typed and 
handwritten user input (the handwriting and typed keyboard input systems are simultaneously 
active and capable of data entry) (Skinner: column 2, line 41 -column 3, Une 3). It would have 
been obvious to one of ordinary skill in the art, having the teachings of Partanen and Skinner 
before him at the time the invention was made, to modify the handwriting recognition method of 
Partanen to include the simultaneous handwriting and typed keyboard input capabilities of 
Skinner. One would have been motivated to make such a combination in order to provide a 
flexible user data entry system, giving the user an option to choose the more convenient input 
mechanism depending on the function to be executed without the hassle of having to switch 
between the input systems. 
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Referring to claims 2 and 19, Partanen, as modified, teach the visible user input interface 
is semi-transparent (Partanen: page 1, paragraphs 0015-0016). 

Referring to claim 3, Partanen, as modified, teach wherein the handwritten data received 
at the input interface is evaluated to determine whether the handwritten data corresponds to a 
gesture (deciphering whether user input is a gesture, such as user selection of virtual buttons and 
keys from a virtual keyboard, or user input of written manuscript characters that requires 
handwriting recognition) (Partanen: pages 2-3, paragraphs 0032-0040). 

Referring to claim 4, Partanen, as modified, teach wherein the handwritten data 
corresponds to a gesture, and further comprising, providing at least one pen event corresponding 
to the gesture to the program (for example, if the handwritten data is a gesture of user selection 
of a virtual button 44 with an external input device such as the pen 30 shown in Figure 2, then 
the pen selection of virtual button 44 causes an event such as deactivation of the semi-transparent 
window) (Partanen: page 3, paragraphs 0038-0040). 

Referring to claims 6 and 26, Partanen, as modified, teach providing the handwritten data 
to a recognition engine in response to detection of a submit button associated with the visible 
user interface (the semi-transparent user interface provides for the display of a virtual keyboard 
having letters, or buttons, the users can select; ftirthermore, user handwritten data is recognized 
after the user gesture of touching a field with the stylus to indicate handwriting and causing 
display of the semi-transparent window) (Partanen: page 1, paragraphs 0014-0016, page 2, 
paragraph 0032 and page 3, paragraphs 0035 and 0038). 

Referring to claim 7, Partanen, as modified, teach providing the handwritten data to a 
recognition engine is performed in response to a time being achieved (displaying the handwritten 
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input for a predetermined period of time after the input from the input device, upon which the 
pattern is completed and recognized by the recognition software) (Partanen: page 2, paragraph 
0024 and page 3, paragraphs 0033-0035). 

Referring to claim 8, Partanen, as modified, teach wherein providing the handwritten data 
to a recognition engine is performed in response to a gesture being detected (user handwritten 
data is recognized after the user gesture of touching a field with the stylus to indicate 
handwriting and causing display of the semi-transparent window) (Partanen: page 1, paragraphs 
0014-0016 and page 2, paragraph 0032). 

Referring to claims 9 and 20, Partanen, as modified, teach evaluating at least one window 
attribute corresponding to the field against hard-coded or retrieved information to determine 
whether the field is supported (evaluating, or determining the window attribute of whether the 
user has contacted the displayed window, or screen, with the stylus) (Partanen: page 1, paragraph 
0016 and page 2, paragraph 0032). 

Referring to claim 10, Partanen, as modified, teach accessing window class information 
(accessing window information such as determining whether user has contacted the windowed 
area with a stylus) (Partanen: page 1, paragraph 0016 and page 2, paragraphs 0027-0030 and 
0032). 

Referring to claim 12, Partanen, as modified, teach adjusting the appearance of the visible 
input window (the semi-transparent window may be moved or sized according to the preferences 
of the user) (Partanen: page 1, paragraph 0015). 
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Referring to claim 13, Partanen, as modified, teach increasing the size of the visible input 
window to enable entry of additional handwritten data (increasing the size of the semi- 
transparent window) (Partanen: page 2, paragraph 0031). 

Referring to claim 14, Partanen, as modified, teach erasing the visible input window 
(selection of the virtual button 44 will cause the semi-transparent window to de-activate and 
disappear) (Partanen: page 3, paragraph 0039). 

Referring to claim 15, Partanen, as modified, teach the visible input window is erased in 
response to a close request (selection of the virtual button 44 will cause the semi-transparent 
window to de-activate and disappear) (Partanen: page 3, paragraph 0039). 

Referring to claim 17, Partanen, as modified, teach wherein the visible input window is 
erased in response to a gesture being detected (upon user gesture of selection of virtual button 
44, the semi-transparent input window is deactivated and disappears) (Partanen: page 3, 
paragraph 0039). 

Referring to claim 21, Partanen, as modified, teach wherein the entered data comprises 
handwritten data (user input of handwritten characters) (Partanen: page 2, paragraph 0022 and 
Figure 1), and further comprising a gesture detection engine that evaluates the handwritten data 
to determine whether the handwritten data corresponds to a gesture (deciphering whether user 
input is a gesture, such as user selection of virtual buttons and keys from a virtual keyboard, or 
user input of written manuscript characters that requires handwriting recognition) (Partanen: 
pages 2-3, paragraphs 0032-0040), and if so, to provide at least one event to the program (for 
example, if the handwritten data is a gesture of user selection of a virtual button 44 with an 
external input device such as the pen 30 shown in Figure 2, then the pen selection of virtual 
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button 44 causes an event such as deactivation of the semi-transparent window) (Partanen: page 
3, paragraphs 0038-0040). 

Referring to claim 23, Partanen, as modified, teach wherein the entered data comprises 
handwritten data (user input of handwritten characters) (Partanen: page 2, paragraph 0022 and 
Figure 1), and further comprising a rulebase that determines an appearance of the visible input 
area including a displayed size thereof (pre-selected preferences that may be established such as 
the display size of the semi-transparent window) (Partanen: page 2, paragraphs 0027-0031). 

Referring to claim 25, Partanen, as modified, teach wherein the visible input area has at 
least one button associated therewith for receiving a command (the semi-transparent window has 
a plurality of buttons that can be selected by the user) (Partanen: page 3, paragraphs 0038-0039 
and further shown in Figure 1). 

Referring to claim 27, Partanen, as modified, teach wherein the user input interface code 
provides the recognition result to the program in a message queue associated with the program 
(as each handwriting character is written, the display signal for displaying the recognized 
handwriting is generated and each characters is displayed on the screen in a queue, i.e. one at a 
time) (Partanen: page 2, paragraphs 0024-0025 and further shown in Figure 1). 

Referring to claim 28, Partanen, as modified, teach wherein the drawing of the visible 
input area positions the visible input area relative to the field based on the information received 
from the field typing engine (the semi-transparent window is displayed at a particular position 
and in a particular size corresponding to user pre-selected preferences) (Partanen: page 2, 
paragraphs 0027-0031). 
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Referring to claim 29, Partanen, as modified, teach wherein the drawing of the visible 
input area sizes the visible input area based on the information received from the field typing 
engine (the semi-transparent window is displayed at a particular position and in a particular size 
corresponding to user pre-selected preferences) (Partanen: page 2, paragraphs 0027-0031). 

3. Claims 5, 16 and 22 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Partanen et al. U.S. PubHcation 2003/0001899 (hereinafter "Partanen") and Skinner U.S. Patent 
6,661,920, as apphed to claims 1 and 18 above, and Frink et al. U.S. Patent 5,956,423 
(hereinafter "Frink"). 

Referring to claims 5 and 22, Partanen and Skinner teach all of the limitations as applied 
to claims 1 and 18 above. Specifically, Partanen and Skinner teach a semi-transparent user 
interface and user input gestures (Partanen: page 1, paragraph 0016, page 3, paragraphs 0038- 
0040 and fiirther shown in Figure 2). However, Partanen and Skinner fail to explicitly teach the 
gesture comprising user input directed to an area of the program that is visible through the user 
interface. Frink teaches an interface for inputting and recognizing handwritten data (Frink: 
column 2, line 37-column 3, line 4) similar to that of Partanen and Skinner. In addition, Frink 
further teaches user inputting gestures directed to an area of the program that is visible through 
the interface (Frink: column 2, line 37-column 3, Une 4 and column 3, lines 52-64). It would 
have been obvious to one of ordinary skill in the art, having the teachings of Partanen, Skinner 
and Frink before him at the time the invention was made, to modify the handwriting input 
interface comprising gesture detection of Partanen and Skinner to include the input of gestures 
taught by Frink. One would have been motivated to make such a combination in order to allow 
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users to easily edit the documents, reducing the confusion of mixing up editing commands and 
data input by the user. 

Referring to claim 16, Partanen, as modified, teach erasing the input window in response 
to a time being achieved (sending the handwritten data to the recognition engine, and therefore 
erasing the input, when the user stops writing for a period of time) (Frink: column 2, lines 44-63 
and Figures 2A, 2B and 2C). 

4. Claim 1 1 is rejected under 35 U.S.C. 103(a) as being unpatentable over Partanen et al. 
U.S. Publication 2003/0001899 (hereinafter "Partanen") and Skinner U.S. Patent 6,661,920, as 
appUed to claim 1 above, and Gilai et al U.S. Patent 6,018,736 (hereinafter "Gilai"). 

Referring to claim 1 1, Partanen and Skinner teach all of the limitations as applied to 
claim 1 above. Specifically, Partanen and Skinner teach obtaining information indicative of 
whether the field is configured to receive text input (determining whether the user has initiated 
manuscript input to any point on a touch-activated screen of a display, thereby causing a semi- 
transparent window to be opened) (Partanen: page 1, paragraph 0016). However, Partanen and 
Skinner fail to explicitly teach accessing a database to obtain information. Gilai teaches a 
method comprising handwriting recognition (Gilai: column 23, lines 7-10) similar to that of 
Partanen and Skinner. In addition, Gilai fiirther teaches accessing a database to obtain 
information (Gilai: column 5, line 63 - column 6, line 3). It would have been obvious to one of 
ordinary skill in the art, having the teachings of Partanen, Skinner and Gilai before him at the 
time the invention was made, to modify the method of determining whether a field is configured 
to receive text input of Partanen and Skinner to include the access of a database to obtain 
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information taught by Gilai. One would have been motivated to make such a combination in 
order to provide an organized and efficient way of indexing, storing and retrieving a large 
amount of information. 

5. Claim 24 is rejected under 35 U.S.C. 103(a) as being unpatentable over Partanen et al. 
U.S. PubUcation 2003/0001899 (hereinafter "Partanen"), Skinner U.S. Patent 6,661,920, as 
applied to claim 18 above, and Microsoft® Excel, copyright 1999 (hereinafter "Excel"). 

Referring to claim 24, Partanen and Skinner teach all of the limitations as applied to 
claim 18 above. However, although Partanen and Skinner teach increasing the size of the visible 
input window (re-sizing the displayed semi-transparent window) (Partanen: page 2, paragraphs 
0027-0031), Partanen and Skinner fail to explicitly teach increasing the size of the visible input 
window based on the data approaching an end thereof and to enable entry of additional data. 
Excel teaches an input interface that adjusts the appearance of the visible input window (see 
screenshots 2 and 3 attached at the end of the office action) similar to that of Partanen and 
Skinner. In addition. Excel ftirther teaches increasing the size of the visible input window based 
on the data approaching an end thereof and to enable entry of additional data (see screenshots 2 
and 3). It would have been obvious to one of ordinary skill in the art, having the teachings of 
Partanen, Skinner and Excel at the time the invention was made, to modify the input interface of 
Partanen and Skinner to include increasing the size of the input window as needed, taught by 
Excel. One would have been motivated to make such a combination in order to allow users to 
enter as much information as needed, making it easier for them to input and view information. 
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6. Claims 30-33 are rejected under 35 U.S.C. 103(a) as being unpatentable over Partanen et 
aL U.S. Publication 2003/0001899 (hereinafter "Partanen"), Skinner U.S. Patent 6,661,920 and 
Frink et al. U.S. Patent 5,956,423 (hereinafter "Frink"). 

Referring to claim 30, Partanen teaches a system comprising an application program 
having at least one application input area into which user input data can be entered (the email 
program shown in Figure 1 has a plurality of user input areas such as Attachments, Subject, etc., 
into which user data can be entered) (Partanen: page 1, paragraphs 0014-0016); user interface 
code external to the application program (semi-transparent interface window for receiving hand- 
written input) (Partanen: pages 1-2, paragraph 0022, pages 2-3, paragraphs 0032-0040 and 
ftirther shown in Figure 1); a typing engine that determines whether to call the user interface 
code for a selected appUcation input area of the appUcation program based on attribute 
information associated with that application input area (based on attribute information for the 
application input area, such as whether user has initiated manuscript input to a point of the input 
area on a touch-activated screen, it is determined whether the user interface code should be 
called, i.e. whether a semi-transparent window should be displayed) (Partanen: page 1, paragraph 
0016, pages 1-2, paragraph 0022 and pages 2-3, paragraphs 0032-0040), the user interface code 
providing a semi-transparent input area based on the attribute information when called, the semi- 
transparent input area configured such that a user may still enter data via user input into the user 
interface code (upon user initiation of manuscript input at point 36 on the display screen of 
Figure 1, a semi-transparent window is displayed relative to the field, allowing users to input 
manuscript characters, for example, the letter "I", while the field is operable to receive input data 
such as user input of particular letters and/or numbers from a virtual keyboard) (Partanen: page 
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2, paragraph 0032 and page 3, paragraph 0038); a gesture engine, the gesture engine invoked to 
determine whether user input data directed to the semi-transparent input area is a gesture directed 
to the appHcation program or information that should be recognized as text (deciphering whether 
user input is a gesture, such as user selection of virtual buttons and keys from a virtual keyboard, 
or user input of written manuscript characters that requires handwriting recognition) (Partanen: 
pages 2-3, paragraphs 0032-0040); and a handwriting recognition engine, the handwriting 
recognition engine configured to receive the information that the gesture engine has decided 
should be recognized as text (when users input handwritten character, the pattern of the 
handwritten is relayed to the handwriting recognition software) (Partanen: page 2, paragraph 
0025), the handwriting recognition engine responding by returning recognized text when 
provided with the information (for example, recognizing the handwritten "I" and outputting it as 
the character "I" in Figure 1) (Partanen: page 2, paragraph 0022). However, ahhough Partanen 
teaches a virtual keyboard, Partanen fails to explicitly teach simultaneously receiving typed and 
handwritten user input. Skinner teaches an interface that allows users to enter handwritten data 
similar to that of Partanen. In addition, Skinner further teaches receiving typed user input and 
the program field being operable to simultaneously receive typed and handwritten user input (the 
handwriting and typed keyboard input systems are simultaneously active and capable of data 
entry) (Skinner: column 2, line 41 -column 3, line 3). It would have been obvious to one of 
ordinary skill in the art, having the teachings of Partanen and Skinner before him at the time the 
invention was made, to modify the handwriting recognition method of Partanen to include the 
simultaneous handwriting and typed keyboard input capabilities of Skinner. One would have 
been motivated to make such a combination in order to provide a more flexible user data entry 
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system, giving the user an option to choose the more convenient input mechanism depending on 
the function to be executed without the hassle of having to switch between the input systems. 
However, although Partanen and Skinner teach a timing mechanism (displaying the handwritten 
input for a predetermined period of time after the input from the input device, upon which the 
pattern is completed and recognized by the recognition software) (Partanen: page 2, paragraph 
0024 and page 3, paragraphs 0033-0035), Partanen and Skinner fail to explicitly teach a timing 
mechanism configured to cause removal of the semi-transparent input area when no user 
interaction with the visible input area is detected for a period of time. Frink teaches an interface 
for inputting and recognizing handwritten data (Frink: column 2, Unes 37-65) similar to that of 
Partanen and Skinner. In addition, Frink ftjrther teaches a timing mechanism configured to cause 
removal of the input when no user interaction with the input area is detected for a period of time 
(sending the handwritten data to the recognition engine, and therefore erasing the input, when the 
user stops writing for a period of time) (Frink: column 2, lines 44-63 and Figures 2A, 2B and 
2C). It would have been obvious to one of ordinary skill in the art, having the teachings of 
Partanen, Skinner and Frink before him at the time the invention was made, to modify the input 
recognition interface comprising the timing mechanism of Partanen and Skinner to include the 
timing mechanism taught by Frink. One would have been motivated to make such a combination 
in order to allow users to easily edit the documents, reducing the conftision of mixing up editing 
commands and data input by the user. Furthermore, it allows users to perform fimctions such as 
note taking faster and more efficiently; recognizing characters after a certain time has elapsed, 
representing the user has completed taking notes, is faster and more efficient than translating the 
written data character by character as the user is taking notes. 
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Referring to claim 31, Partanen, as modified, teach wherein the recognized text is 
received by the user interface code and made available to the application program (handwritten 
text characters are input through the semi-transparent window and recognized text are displayed 
on the email program shown in Figure 1) (Partanen: page 2, paragraphs 0022-0025 and 0032). 

Referring to claim 32, Partanen, as modified, teach wherein the application program 
displays the recognized text in the application input area (text characters are displayed on the 
semi-transparent window and on area 20 of Figure 1) (Partanen: page 2, paragraphs 0022-0025 
and 0032). 

Referring to claim 33, Partanen, as modified, teach a growth rulebase, the growth 
rulebase determining whether to alter an appearance of the semi-transparent input area in 
response to the information received therein (pre-selected preferences that may be established 
such as the display size of the semi-transparent window can be altered) (Partanen: page 2, 
paragraphs 0027-0031). 

Response to Arguments 

7. Applicant's arguments with respect to claims 1, 18 and 30 have been considered but are 
moot in view of the new ground(s) of rejection. However, in the interest of furthering 
prosecution, several of the applicant's arguments will be addressed below: 

8. With respect to claims 1,18 and 30, the applicant argues that the input of data from a 
virtual keyboard of Partanen is not a form of typed input. The examiner respectfully disagrees. 
The definition of "type" from Microsoft's Computer Dictionary Fifth Edition (copy of which is 
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attached with the office action) is defined to be "to enter information by means of the keyboard". 
The definition does not specify that the keyboard is an actual physical keyboard instead of a 
virtual keyboard. In addition, the examiner respectfially points out that the language of the 
recited claims does not explicitly specify that the typed input data is received from a physical 
keyboard. Therefore, the examiner argues that Partanen's teaching of entering information via 
the keyboard, albeit the virtual keyboard, is a form of typed input. Similarly, Skinner's teachings 
of simultaneously receiving input from the handwriting input system and the virtual keyboard 
system, as recited in column 2, Hne 41 -column 3, line 3, teaches simultaneously receiving typed 
input data and handwritten input data. 

9. With respect to claim 10, the applicant argues that Partanen simply teaches expanding an 
input field when double-tapped and does not show any capability of assessing window attributes 
or window class information. The examiner respectfully disagrees. Partanen teaches that 
window attributes such as whether the user has contacted the displayed window with the stylus, 
are evaluated in order to display a semi-transparent window; furthermore, when it is determined 
that the user has contacted the window, a signal is generated to the data processor to access semi- 
transparent window's attributes, or class information, i.e. the preferences established for opening 
the semi-transparent window, as recited on page 1, paragraph 0016 and page 2, paragraph 0032. 
Therefore, the examiner respectfully maintains that Partanen teaches assessing window attributes 
or window class information. 



Application/Control Number: 09/976, 1 88 Page 1 6 

Art Unit: 2173 

10. Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706,07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1. 136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1. 136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Ting Zhou whose telephone number is (571) 272-4058. The 
examiner can normally be reached on Monday - Friday 7:00 am - 4:30 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Cabeca can be reached at (571) 272-4048. The fax phone number for the 
organization where this application or proceeding is assigned is (571) 273-8300. 
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